Add context key support for strict model aliases and enhance logging#2
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR expands LocalBox’s model launch/alias system to support context-aware strict aliases, introduces optional vision (mmproj) support across the wizard/shortcuts, and adds a dedicated launch debug log for troubleshooting.
Changes:
- Add
-ContextKeysupport for strict alias naming/building and propagate it through init/orphan management paths. - Add
-UseVisionflow (wizard + shortcuts) and HuggingFace mmproj discovery/download to enable multimodal launches. - Add launch/debug logging (
~/.local-llm/launch.log,llmlog) and extend llama.cpp tuning/args surfaces.
Reviewed changes
Copilot reviewed 18 out of 19 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents new llmlog launch log and troubleshooting guidance. |
| .gitignore | Ignores *.log files. |
| local-llm/lib/99-entrypoints.ps1 | Forwards CLI args to Start-LLMWizard so llm -UseVision works. |
| local-llm/lib/90-wizard.ps1 | Adds vision prompts + passes -UseVision through selection/launch flows; adds launch log helpers. |
| local-llm/lib/85-shortcuts.ps1 | Adds -UseVision switch and threads context-aware strict alias selection. |
| local-llm/lib/80-init.ps1 | Rebuilds strict aliases with context keys during stale rebuilds. |
| local-llm/lib/75-display.ps1 | Adds VisionModule display and registers llmlog in command reference; logs Spectre import failures to launch log. |
| local-llm/lib/72-llamacpp-tuner.ps1 | Extends override formatting + adds NCpuMoeCandidates parameter. |
| local-llm/lib/71-benchpilot-bridge.ps1 | Adds NCpuMoeCandidates plumbing and helper to fetch top NCpuMoe values; logs failures to launch log. |
| local-llm/lib/65-claude-launch.ps1 | Adds vision resolution/logging for llama.cpp launches and sets context token env vars; adds launch trace logging. |
| local-llm/lib/60-catalog.ps1 | Adds mmproj discovery/prompting to addllm/updatellm; expands managed strict alias names per context. |
| local-llm/lib/55-huggingface.ps1 | Adds mmproj discovery via HF API and extra logging; adjusts HF HTTP calls. |
| local-llm/lib/50-modelfile.ps1 | Adds VISION lines to generated Modelfiles and makes strict aliases context-aware. |
| local-llm/lib/41-llamacpp-args.ps1 | Adds llama-server args for --mmproj, --swa-full, and --cache-prompt. |
| local-llm/lib/40-parsers.ps1 | Updates strict Modelfile comment to reflect selected base context behavior. |
| local-llm/lib/35-backend.ps1 | Threads -UseVision through backend dispatcher. |
| local-llm/lib/20-models.ps1 | Implements mmproj auto-detection/download and availability probing. |
| local-llm/lib/10-helpers.ps1 | Adjusts download behavior for HF files (including TLS handling changes). |
| local-llm/lib/00-settings.ps1 | Adds Claude context token env var names to backup/restore list. |
Comments suppressed due to low confidence (5)
local-llm/lib/65-claude-launch.ps1:616
- This string interpolation is malformed:
"Launching ${$backendLabel} ..."uses${$backendLabel}, which will try to treat the value of$backendLabelas a variable name (and typically prints blank or errors). Use$backendLabel/${backendLabel}instead.
This issue also appears on line 1027 of the same file.
Env = $env
LaunchExe = $launchExe
LaunchArgs = $launchExeArgs
local-llm/lib/90-wizard.ps1:1266
- In the
setupaction, the strict branch callsEnsure-ModelStrictAlias -Key $ModelKey -ForceRebuildbut does not pass-ContextKey $ContextKey. With context-aware strict aliases, this will rebuild only the base strict alias rather than the selected context-specific strict alias.
"setup" {
$defSetup = Get-ModelDef -Key $ModelKey
$visionSetup = if ($UseVision) { Get-ModelVisionModulePath -Key $ModelKey -Def $defSetup -Backend ollama } else { '' }
$modelName = if ($Strict) { Ensure-ModelStrictAlias -Key $ModelKey -ForceRebuild } else { Ensure-ModelAlias -Key $ModelKey -ContextKey $ContextKey -ForceRebuild -VisionModulePath $(if ($visionSetup) { $visionSetup } else { '' }) }
local-llm/lib/90-wizard.ps1:1273
- In the
showaction, the strict branch callsEnsure-ModelStrictAliaswithout-ContextKey $ContextKey, soollama showwon’t reflect the context-specific strict alias when a non-default context is selected. Pass the context key to keep behavior consistent with context-aware strict aliases.
"show" {
$defShow = Get-ModelDef -Key $ModelKey
$visionShow = if ($UseVision) { Get-ModelVisionModulePath -Key $ModelKey -Def $defShow -Backend ollama } else { '' }
$modelName = if ($Strict) { Ensure-ModelStrictAlias -Key $ModelKey } else { Ensure-ModelAlias -Key $ModelKey -ContextKey $ContextKey -VisionModulePath $(if ($visionShow) { $visionShow } else { '' }) }
local-llm/lib/85-shortcuts.ps1:70
- In the strict branch,
Ensure-ModelStrictAliasis called without any vision-related input, but-UseVisionis accepted and the code computes$visionModulePathfor non-strict launches only. Combined withEnsure-ModelStrictAliascurrently embedding vision by default, this makes-UseVisionbehavior inconsistent between strict vs non-strict. Consider threading a vision choice into strict alias creation (or explicitly documenting/enforcing that strict aliases are always vision-enabled when available).
# Resolve vision module (mmproj) on demand when user opts in; always log availability.
$visionModulePath = if ($UseVision) {
Write-LaunchLog "Resolving vision module for Ollama launch (model=$Key)" 'VISION'
$result = Get-ModelVisionModulePath -Key $Key -Def $def -Backend ollama
if ($result) {
Write-LaunchLog "Vision module resolved: $([System.IO.Path]::GetFileName($result))" 'VISION'
} else {
Write-LaunchLog "No vision module found for $Key (Ollama)" 'WARN'
}
$result
} else {
$avail = Test-ModelVisionModuleAvailable -Key $Key -Def $def -Backend ollama
if ($avail.Local) {
Write-LaunchLog "Vision available locally ($($avail.Filename)) — not loaded (no -UseVision)" 'VISION'
} elseif ($avail.AvailableOnHF) {
Write-LaunchLog "Vision available on HuggingFace ($($avail.Filename)) — not loaded (no -UseVision)" 'VISION'
} else {
Write-LaunchLog "No vision module available for $Key (Ollama)" 'VISION'
}
''
}
# DryRun resolves the alias name without rebuilding the Modelfile (which
# would create a side-effecting `ollama create`). Real launches still call
# Ensure-Model* so a stale or missing alias is rebuilt on the spot.
$modelName = if ($DryRun) {
local-llm/lib/65-claude-launch.ps1:1029
Write-LaunchLog "Launching ${$backendLabel}: ..."uses${$backendLabel}, which treats the value of$backendLabelas a variable name rather than interpolating the label. This will usually log an empty string for the backend label. Use$backendLabel/${backendLabel}instead.
$launchExe = if ($Unshackled) { 'unshackled' } elseif ($Codex) { 'codex' } else { 'claude' }
$launchExeArgs = if ($Codex) {
@()
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
23
to
+27
| param([Parameter(Mandatory = $true)][string]$Repo) | ||
|
|
||
| $url = "https://huggingface.co/api/models/$Repo`?blobs=true" | ||
| return Invoke-RestMethod -Uri $url -UseBasicParsing | ||
| return Invoke-RestMethod -Uri $url -UseBasicParsing -SkipCertificateCheck | ||
| } |
Comment on lines
124
to
129
| $url = "https://huggingface.co/$Repo/raw/main/README.md" | ||
|
|
||
| try { | ||
| $resp = Invoke-WebRequest -Uri $url -UseBasicParsing -TimeoutSec 10 | ||
| $resp = Invoke-WebRequest -Uri $url -UseBasicParsing -TimeoutSec 10 -SkipCertificateCheck | ||
| return [string]$resp.Content | ||
| } catch { |
Comment on lines
93
to
98
|
|
||
| try { | ||
| $env:PYTHONUTF8 = "1" | ||
| $env:PYTHONIOENCODING = "utf-8" | ||
| $env:HF_HUB_DISABLE_SSL_VERIFICATION = "1" | ||
|
|
Comment on lines
152
to
157
| $request = [System.Net.HttpWebRequest]::Create($url) | ||
| $request.Method = "GET" | ||
| $request.AllowAutoRedirect = $true | ||
| $request.UserAgent = "LocalLLMProfile/1.0" | ||
| $request.ServerCertificateValidationCallback = { $true } | ||
|
|
Comment on lines
+262
to
+272
| # Check for mmproj (multimodal vision module). Prompt the user to download. | ||
| if ($PSBoundParameters.ContainsKey('Mmproj')) { | ||
| $mmprojFile = [string]$Mmproj | ||
| if (-not [string]::IsNullOrWhiteSpace($mmprojFile)) { | ||
| $entry.VisionModule = $mmprojFile | ||
| } | ||
| } | ||
| else { | ||
| $mmprojFiles = Get-HuggingFaceMmprojFiles -Repo $repo | ||
| if ($mmprojFiles.Count -gt 0) { | ||
| $mmprojNames = @($mmprojFiles.Keys) -join ', ' |
Comment on lines
18
to
+23
| $content.Add("FROM $FromSource") | ||
|
|
||
| if (-not [string]::IsNullOrWhiteSpace($VisionModulePath)) { | ||
| $content.Add("VISION $VisionModulePath") | ||
| } | ||
|
|
Comment on lines
176
to
182
| function Ensure-ModelAlias { | ||
| param( | ||
| [Parameter(Mandatory = $true)][string]$Key, | ||
| [Parameter(Mandatory = $true)][AllowEmptyString()][string]$ContextKey, | ||
| [switch]$ForceRebuild | ||
| [switch]$ForceRebuild, | ||
| [string]$VisionModulePath | ||
| ) |
Comment on lines
225
to
+237
| $def = Get-ModelDef -Key $Key | ||
| $strictName = Get-ModelStrictAliasName -Def $def | ||
| $baseCtxKey = Get-ModelStrictBaseContextKey -Def $def | ||
| $baseCtxKey = if ([string]::IsNullOrWhiteSpace($ContextKey)) { | ||
| Get-ModelStrictBaseContextKey -Def $def | ||
| } else { | ||
| Resolve-ModelContextKey -Def $def -ContextKey $ContextKey | ||
| } | ||
| $strictName = Get-ModelStrictAliasName -Def $def -ContextKey $baseCtxKey | ||
| $baseName = Get-ModelAliasName -Def $def -ContextKey $baseCtxKey | ||
| $numCtx = Get-ModelContextValue -Def $def -ContextKey $baseCtxKey | ||
|
|
||
| # Resolve vision module so the strict alias gets an explicit VISION instruction. | ||
| $visionModulePath = Get-ModelVisionModulePath -Key $Key -Def $def -Backend ollama | ||
|
|
Comment on lines
256
to
261
| $def = Get-ModelDef -Key $Key | ||
| $visionModulePath = Get-ModelVisionModulePath -Key $Key -Def $def -Backend ollama | ||
|
|
||
| foreach ($contextKey in $def.Contexts.Keys) { | ||
| Ensure-ModelAlias -Key $Key -ContextKey $contextKey -ForceRebuild:$ForceRebuild | Out-Null | ||
| Ensure-ModelAlias -Key $Key -ContextKey $contextKey -ForceRebuild:$ForceRebuild -VisionModulePath $visionModulePath | Out-Null | ||
| } |
|
|
||
| $sizeGB = if ($bytes -gt 0) { [math]::Round($bytes / 1000000000, 1) } else { 0 } | ||
| $map[$name] = $sizeGB | ||
| Write-LaunchLog "Found mmproj: $name$(if ($sizeGB -gt 0) { " ($sizeGB GB)" })" 'INFO' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.